Search for: All records

Today’s generative models can synthesize magnetic resonance images (MRIs) of the brain at specific ages. However, such models can neither map the aging process longitudinally within subjects, nor accommodate its variability across subjects. Such approaches also cannot predict anatomic features of aging in ways that can be validated retrospectively or trusted prospectively. We introduce a three-dimensional hybrid ControlNet + diffusion model that uses the baseline T1-weighted MRIs of healthy adults to predict individual neuroanatomic aging trajectories, as reflected by follow-up MRIs. The approach captures individual anatomical changes with an average predicted voxelwise intensity error of 15% and structural similarity index of 93%. Unlike methods relying on qualitative validation, our approach quantifies the fidelity of prospective MRI synthesis using FreeSurfer volumetrics. Because brain atrophy reflects risk for Alzheimer’s disease (AD), our model’s ability to generate individual-specific prospective MRIs suggests its clinical potential to assist AD risk estimation. 

Integrating and processing information from various sources or modalities are critical for obtaining a comprehensive and accurate perception of the real world in autonomous systems and cyber-physical systems. Drawing inspiration from neuroscience, we develop the Information-Theoretic Hierarchical Perception (ITHP) model, which utilizes the concept of information bottleneck. Different from most traditional fusion models that incorporate all modalities identically in neural networks, our model designates a prime modality and regards the remaining modalities as detectors in the information pathway, serving to distill the flow of information. Our proposed perception model focuses on constructing an effective and compact information flow by achieving a balance between the minimization of mutual information between the latent state and the input modal state, and the maximization of mutual information between the latent states and the remaining modal states. This approach leads to compact latent state representations that retain relevant information while minimizing redundancy, thereby substantially enhancing the performance of multimodal representation learning. Experimental evaluations on the MUStARD, CMU-MOSI, and CMU-MOSEI datasets demonstrate that our model consistently distills crucial information in multimodal learning scenarios, outperforming state-of-the-art benchmarks. Remarkably, on the CMU-MOSI dataset, ITHP surpasses human-level performance in the multimodal sentiment binary classification task across all evaluation metrics (i.e., Binary Accuracy, F1 Score, Mean Absolute Error, and Pearson Correlation).